Recently, multi-view coded video (MVC) has been widely recognized as one of the more promising trends of video oriented applications, and became an important focus in many video related industries, for example, entertainment, TV, multimedia, and so forth. Multi-view video coding (MVC) is the compression framework for the encoding of multi-view sequences. A Multi-view Video Coding (MVC) sequence is a set of two or more video sequences that capture the same scene from a different view point. When looking for standards in which to code multi-view content, one such available standard is the MVC extension of the International Organization for Standardization/International Electrotechnical Commission (ISO/IEC) Moving Picture Experts Group-4 (MPEG-4) Part 10 Advanced Video Coding (AVC) Standard/International Telecommunication Union, Telecommunication Sector (ITU-T) H.264 Recommendation (hereinafter the “MPEG-4 AVC Standard”). Furthermore, development of other advanced multi-view video coding standards are also currently being explored and developed.
In a multi-view video coding context, the goal of rate control (RC) becomes more complicated and complex than that of the conventional single view two-dimensional (2D) video coding. In the single view context, one needs to optimize the coding performance of one set of two-dimensional images only. However, in the case of multi-view video coding, one needs to not only optimize the overall coding performance of all the views together (referred to interchangeably herein as the “joint view” or “joint views”), but also needs to satisfy other possible bit rate constraints from particular views separately and in addition to the joint view. In particular, one example of such single view requirements is the separate and explicit bit rate requirements for the base view video. With such single view requirements, one can ensure some guarantee on the base view coding quality, which is highly desirable and important in practice. Since there will still be quite a lot customers using conventional two-dimensional video services who will see the base view video only, a service provider has to make sure the new multi-view video coding service will not compromise their quality of service for the two-dimensional video customers. How to balance the goal of joint view optimality with the base view constraint is a new rate control problem for multi-view video coding.
Existing rate control schemes all target conventional single view 2D video coding. Different rate control schemes can be categorized by their adopted different bit rate models, i.e., R-QP relationships; where R denotes the bit rate of a coding unit such as, for example, a frame, macroblock, and so forth, and QP denotes the quantization parameter of the coding unit. A well-known and widely used model is a quadratic R-QP model, for example, used in MPEG-2 Test Model Version 5 (TM5). Another model is ITU H.263 Test Model, Near Term Version 8 (TMN8) rate control. Additionally, and more recently, improved and more accurate R-ρ-QP models have been introduced, i.e., the so called p-domain model. In addition, rate control schemes can also be categorized by (i) targeting constant bit rate (CBR) coding or variable bit rate (VBR) coding, and (ii) targeting real-time coding and hence preferring single-pass coding or offline coding and thus allowing for multi-pass coding, and so forth.
We emphasize that beyond all these type differences, all these existing schemes were originally developed for single-view video coding, which optimizes the coding performance of one single view of 2D video only. However, the emergence of MVC video poses a new and more complicated rate control problem, due to the presence of multiple views of 2D videos to be coded at the same time. Specifically, besides the goal of performance optimality of the joint view, i.e., with all the single views included, some views may have their own coding constraints in particular. Especially for the base view, a separate set of coding bit rate constraints has to be met in practice to ensure the new multi-view video coding service will not yield degraded quality of service for the legacy two-dimensional video coding.